Multi-aperture camera system having auto focusing function and/or depth estimation function

ABSTRACT

A multi-aperture camera system having an auto focusing function comprises: multiple apertures; an image sensor that creates multiple images by processing light signals introduced through the multiple apertures, respectively; and an auto focusing unit that determines a distance by which the image sensor moves relative to the multiple apertures by using the multiple images for auto focusing, wherein at least one of the multiple apertures has a central position that is misaligned with those of the remaining apertures other than the at least one aperture.

TECHNICAL FIELD

The present disclosure relates to a multi-aperture camera system havingan auto focusing function and/or a depth estimation function, and moreparticularly, relates to technologies for the multi-aperture camerasystem having the auto focusing function and/or the depth estimationfunction by including a plurality of apertures.

BACKGROUND ART

After cameras as mass-produced models, each of which includes an autofocusing function, have first come to the market in 1977 from Konica, avariety of auto focusing technologies applying mechanical andnon-mechanical systems have been developed. Further, severaltechnologies for depth estimation of subjects using stereo cameratechnology, time of flight technology, structured technology, and thelike may have been developed.

Conventional auto focusing technology may be roughly classified as anactive system or a passive system. Since an ultrasonic or infraredgenerator is needed in the active system divided into a system usingultrasonic waves and a system using infrared rays, relatively largeamounts of additional power may be required. If there is a windowbetween a camera and a subject or if the camera and the subject are in avery distant place or are close to each other, focus may fail to bemade.

The passive system may be an autofocus rangefinder system, a contrastdetection system, a phase-difference detection system, or the like. Theautofocus rangefinder system may be a system of extracting a depththrough a method of performing coincidence of stereo images based on atriangulation principle and performing auto focusing in a rangefindercamera. The contrast detection system may be an auto focusing systemcurrently used in most compact digital cameras and may be a system ofcontinuing calculating contrasts of some of images while moving a lensand determining that focus is made if there is a maximum contrast value.The phase-difference detection system may divide light incident througha lens into one pair and compare a pair of lights to determine whetherfocus is made. For this purpose, a sensor for phase-difference detectionshould be used additionally, or a sensor of a dual pixel structureshould be used.

However, the autofocus rangefinder system should perform auto focusingusing an image obtained by an additional structure which is arangefinder rather than an image captured by a camera. Since thecontrast detection system calculates a contrast between images whilecontinuing moving a lens, it takes a long time to perform auto focusing.If a phase-difference detection sensor is separately used in thephase-difference detection system, since a very precise array is needed,there may be a problem in mass production and quality management and itmay be difficult to be applied to a micro-camera such as a mobile phonecamera. Further, sensor costs may be increased in a system forphase-difference detection using a dual pixel sensor, particularly, amicro-camera have a technical problem in implementing the dual pixelsensor on a trend where an image sensor pixel becomes smaller andsmaller in size.

Stereo camera technology which is one of technologies for depthestimation for a subject may be a manner of installing two opticalsystems at a constant distance, obtaining images of the subject fromsensors of the two optical systems, and comparing a disparity betweenthe two images to estimate a depth. Since this manner obtains imagesusing the two different sensors, a synchronization process between thetwo images may be needed and costs for a precise array and imageadjustment for the synchronization process may be needed.

Further, the time of flight technology and the structured lighttechnology among technologies for depth estimation of a subject may betechnologies of irradiating rays such as infrared rays and estimating adepth using a detector. It may be difficult for the time of flighttechnology and the structured light technology to be used outdoors. Thetime of flight technology and the structured light technology may havemuch power consumption.

Thus, a multi-aperture camera system having an auto focusing functionand/or a depth estimation function for solving disadvantages andproblems of conventional auto focusing technology and/or conventionaldepth estimation technology is proposed.

DETAILED DESCRIPTION OF THE INVENTION Technical Problem

An aspect of the present disclosure is to provide a multi-aperturecamera system having an auto focusing function and/or a depth estimationfunction for solving disadvantages and problems of conventional autofocusing technology and conventional depth estimation technology byincluding a plurality of apertures having different central locations.

Technical Solution

In accordance with an aspect of the present disclosure, a multi-aperturecamera system having an auto focusing function is provided. Themulti-aperture camera system may include a plurality of apertures, animage sensor configured to generate a plurality of images by processingan optical signal introduced through each of the plurality of apertures,and an auto focusing unit configured to determine a moving distance ofthe image sensor using the characteristics of the plurality of aperturesand the plurality of images for auto focusing. At least one apertureamong the plurality of apertures may have a central location which ismisaligned with a central location of the other aperture except for theat least one aperture among the plurality of apertures.

The plurality of apertures may be formed on one optical system.

The at least one aperture may be formed on the other aperture.

The other aperture may be formed by including a filter which detects aspecific optical signal in an optical system in which the plurality ofapertures are formed.

The at least one aperture may be formed by etching a specific region onthe other aperture.

The at least one aperture and the other aperture may be formed to haveone of a circle, a triangle, a quadrangle, or a polygon.

The at least one aperture may introduce an optical signal of a differentwavelength from a wavelength of an optical signal introduced by theother aperture.

The auto focusing unit may determine the moving distance of the imagesensor using the characteristics of the plurality of apertures, based ona difference between central locations of the plurality of images, adistance between a central location of the at least one aperture and acentral location of the other aperture, a distance between an opticalsystem in which the plurality of apertures are formed and the imagesensor, a subject distance which is focused on the image sensor, and afocal distance.

The auto focusing unit may verify the moving distance of the imagesensor using the characteristics of the plurality of apertures and adifference between blur levels in the plurality of images.

In accordance with another aspect of the present disclosure, amulti-aperture camera system having a depth estimation function isprovided. The multi-aperture camera system may include a plurality ofapertures, an image sensor configured to generate a plurality of imagesby processing an optical signal introduced through each of the pluralityof apertures, and a depth estimation unit configured to estimate a depthfrom a subject to an optical system in which the plurality of aperturesare formed, using the plurality of images. At least one aperture amongthe plurality of apertures may have a central location which ismisaligned with a central location of the other aperture except for theat least one apertures among the plurality of apertures.

The plurality of apertures may be formed on one optical system.

The at least one aperture may be formed on the other aperture.

The other aperture may be formed by including a filter which detects aspecific optical signal in the optical system in which the plurality ofapertures are formed.

The at least one aperture may be formed by etching a specific region onthe other aperture.

The at least one aperture and the other aperture may be formed to haveone of a circle, a triangle, a quadrangle, or a polygon.

The at least one aperture may introduce an optical signal of a differentwavelength from a wavelength of an optical signal introduced by theother aperture.

The depth estimation unit may estimate the depth from the subject to theoptical system in which the plurality of apertures are formed, based ona difference between central locations of the plurality of images, adistance between a central location of the at least one aperture and acentral location of the other aperture, a subject distance which isfocused on the image sensor, and a focal distance.

The depth estimation unit may verify the depth from the subject to theoptical system in which the plurality of apertures are formed, using adifference between blur levels in the plurality of images.

Advantageous Effects of the Invention

Embodiments of the present disclosure may provide a multi-aperturecamera system having an auto focusing function and/or a depth estimationfunction for solving disadvantages and problems of conventional autofocusing technology and conventional depth estimation technology byincluding a plurality of apertures having different central locations onone optical system.

In detail, embodiments of the present disclosure may provide amulti-aperture camera system which may be manufactured at a lower costthan a multi-aperture camera system having a conventional auto focusingfunction and a conventional depth estimation function by including aplurality of apertures having different central locations on one opticalsystem.

Further, embodiments of the present disclosure provides a multi-aperturecamera system which may operate an auto focusing function and/or a depthestimation function if a focal distance is long as well as if the focaldistance is short and everywhere in the interior and exterior byincluding a plurality of apertures having different central locations onone optical system.

Further, embodiments of the present disclosure provides a multi-aperturecamera system for performing an auto focusing process and/or a depthestimation process at a high speed by including a plurality of apertureshaving different central locations on one optical system.

DESCRIPTION OF DRAWINGS

FIG. 1 is a drawing illustrating a principle for an auto focusingfunction and a depth estimation function of a camera system according toan embodiment of the present disclosure;

FIG. 2 is a drawing illustrating an auto focusing function and a depthestimation function of a camera system according to an embodiment of thepresent disclosure;

FIG. 3 is a drawing illustrating a plurality of apertures according toan embodiment of the present disclosure;

FIGS. 4A, 4B, and 4C are drawings illustrating a plurality of aperturesaccording to another embodiment of the present disclosure;

FIG. 5 is a flowchart illustrating an auto focusing method of a camerasystem according to an embodiment of the present disclosure;

FIG. 6 is a block diagram illustrating a camera system having an autofocusing function according to an embodiment of the present disclosure;

FIG. 7 is a flowchart illustrating a depth estimation method of a camerasystem according to an embodiment of the present disclosure; and

FIG. 8 is a block diagram illustrating a camera system having a depthestimation function according to an embodiment of the presentdisclosure.

BEST MODE

Hereinafter, a description will be given in detail of embodiments withreference to the accompanying drawings. However, the present disclosureis restricted or limited to embodiments of the present disclosure.Further, like reference numerals shown in each drawing indicate likemembers.

Further, the terminology used in the specification may be terms used toproperly represent an exemplary embodiment of the present disclosure andmay vary according to intention of a user or an operator or custom of afield included in the present disclosure. Therefore, the terminologywill be defined based on contents across the specification.

FIG. 1 is a drawing illustrating a principle for an auto focusingfunction and a depth estimation function of a camera system according toan embodiment of the present disclosure.

Referring to FIG. 1, if an image surface 110 on which an image of asubject is focused is located on location 1 which is a focal distance inthe camera system according to an embodiment of the present disclosure(if an image sensor which generates an image by processing an opticalsignal is located in the image surface 110), since the subject iscorrectly focused, an optical signal introduced through each of aplurality of apertures 120 and 130 may be processed to generate a clearimage. Hereinafter, an embodiment is exemplified as the plurality ofapertures 120 and 130 included in the camera system include the aperture120 which introduces a visible ray and the aperture 130 which introducesan infrared ray. However, embodiments are not limited thereto. Forexample, the plurality of apertures 120 and 130 may include variousapertures which introduce optical signals of different wavelengths.

However, if the image surface 110 is located on location 2, since thesubject is incorrectly focused and a blur occurs, an optical signalintroduced through each of the plurality of apertures 120 and 130 may beprocessed to generate an image in which the blur exists. In this case,since a central location of a visible image generated by the visibleaperture 120 among a plurality of images for the subject is notidentical to a central location of an infrared image generated by theinfrared aperture 130, there may be a phenomenon in which the center ofthe infrared image is skewed towards the right with respect to thecenter of the visible image. On the other hand, if the image surface 110is located on location 3, there may be a phenomenon in which the centerof the infrared image is skewed towards the left with respect to thecenter of the visible image.

The camera system according to an embodiment of the present disclosuremay have an auto focusing function and a depth estimation function usingthe above-mentioned principle by including the plurality of apertures120 and 130, central locations of which are misaligned with each other.A detailed description for this is given hereafter.

FIG. 2 is a drawing illustrating an auto focusing function and a depthestimation function of a camera system according to an embodiment of thepresent disclosure.

Referring to FIG. 2, the camera system according to an embodiment of thepresent disclosure may have the auto focusing function and the depthestimation function by including a plurality of apertures 210 and 220using the above-mentioned principle.

In detail, a disparity p between a central location of a visible imagegenerated by the visible aperture 210 among the plurality of apertures210 and 220 and a central location of an infrared image generated by theinfrared aperture 220 among the plurality of apertures 210 and 220 maybe calculated like Equation 1 based on the above-mentioned principle.

$\begin{matrix}{p = {{\frac{x}{D} \cdot \frac{f^{2}}{F\#( {a_{0} - f} )}}( {\frac{a_{0}}{a} - 1} )}} & \lbrack {{Equation}\mspace{14mu} 1} \rbrack\end{matrix}$

In Equation 1, x may refer to a disparity between the central locationof the visible aperture 210 and the central location of the infraredaperture 220, D may refer to a diameter of the visible aperture 210, fmay refer to a focal distance, F # may refer to a brightness value of alens in the camera system, a may refer to a subject distance, and a₀ mayrefer to a subject distance which is focused on an image sensor 230.

In this case, if a value of the disparity p between the central locationof the visible image and the central location of the infrared image ischanged from a positive number to a negative number or is changed fromthe negative number to the positive number, a direction of a disparitybetween the two images may be changed. Thus, it may be distinguishedwhether a subject having a boundary blur is located in the foreground orbackground, according to a sign of a value of the disparity p.

Further, depth a for the subject (a depth from the subject to an opticalsystem 240 in which the plurality of apertures 210 and 220 are formed)may be calculated like Equation 2 from Equation 1.

$\begin{matrix}{a = \frac{a_{0}}{1 + {\frac{( {a_{0} - f} )}{f} \cdot \frac{p}{x}}}} & \lbrack {{Equation}\mspace{14mu} 2} \rbrack\end{matrix}$

In Equation 2, a₀ may refer to a subject distance which is focused onthe image sensor 230, f may refer to a focal distance, p may refer to adisparity between the central location of the visible image and thecentral location of the infrared image, and x may refer to a disparitybetween the central location of the visible aperture 210 and the centrallocation of the infrared aperture 220.

Further, distance Δb in which the image sensor 230 moves for autofocusing may be calculated like Equation 3.

$\begin{matrix}{{\Delta\; b} = {\frac{b_{0}}{\frac{x}{p} - 1} = {\frac{1}{\frac{x}{p} - 1} \cdot \frac{a_{0}f}{a_{0} - f}}}} & \lbrack {{Equation}\mspace{14mu} 3} \rbrack\end{matrix}$

In Equation 3, P may refer to a disparity between the central locationof the visible image and the central location of the infrared image, xmay refer to a disparity between the central location of the visibleaperture 210 and the central location of the infrared aperture 220, b₀may refer to a distance between the optical system 240 in which theplurality of apertures 210 and 220 are formed and the image sensor 230,a₀ may refer to a subject distance which is focused on the image sensor230, and f may refer to a focal distance.

As such, the camera system according to an embodiment of the presentdisclosure may estimate a depth like Equation 2 and may automaticallyadjust a focus like Equation 3 by including the plurality of apertures210 and 220 formed such that their central locations are misaligned witheach other.

In this case, the plurality of apertures 210 and 220 may be formed onthe one optical system 240. A detailed description for this will begiven with reference to FIGS. 3 to 4C.

Further, the camera system may verify the estimated depth and theadjusted focus like Equations 4 to 6 using a difference between blurlevels in a plurality of images respectively generated by the pluralityof apertures 210 and 220.

First of all, blur d of each of a visible image and an infrared imagemay be represented like Equation 4.

$\begin{matrix}{d = {\frac{f^{2}}{F\#( {a_{0} - f} )}{{\frac{a_{0}}{a} - 1}}}} & \lbrack {{Equation}\mspace{14mu} 4} \rbrack\end{matrix}$

In Equation 4, f may refer to a focal distance, F # may refer to abrightness value of a lens in the camera system, a may refer to a depthto the optical system 240 in which the plurality apertures 210 and 220are formed, and a₀ may refer to a subject distance which is focused onthe image sensor 230.

Thus, the camera system may calculate Equation 5 indicating a depth fromthe subject to the optical system 240 in which the plurality apertures210 and 220 are formed and may calculate Equation 6 indicating adistance Δb in which the image sensor 230 moves for auto focusing.

$\begin{matrix}{a = {\frac{a_{0}}{1 + {\frac{F\#( {a_{0} - f} )}{f^{2}}d}}( {a < a_{0}} )}} & \lbrack {{Equation}\mspace{14mu} 5} \rbrack \\{a = {\frac{a_{0}}{1 + {\frac{F\#( {a_{0} - f} )}{f^{2}}d}}( {a < a_{0}} )}} & \;\end{matrix}$

In Equation 5, F # may refer to a brightness value of the lens in thecamera system, a₀ may refer to a subject distance which is focused onthe image sensor 230, f may refer to a focal distance, and d may referto a level of a blur of each of a visible image and an infrared image ora disparity between blurs of the visible image and the infrared image.Thus, the camera system may verify a depth estimated from Equation 2using Equation 5.

$\begin{matrix}{{\Delta\; b} = {\frac{b_{0}}{{\frac{f}{{dF}\#} - 1}} = {\frac{1}{{\frac{f}{{dF}\#} - 1}} \cdot \frac{a_{0}f}{a_{0} - f}}}} & \lbrack {{Equation}\mspace{14mu} 6} \rbrack\end{matrix}$

In Equation 6, d may refer to a level of a blur of each of the visibleimage and the infrared image or a disparity between blurs of the visibleimage and the infrared image, f may refer to a focal distance, F # mayrefer to a brightness value of the lens in the camera system, b₀ mayrefer to a distance between the optical system 240 in which theplurality of apertures 210 and 220 are formed and the image sensor 230,and a₀ may refer to a subject distance which is focused on the imagesensor 230. Thus, the camera system may verify the distance in which theimage sensor 230 move for auto focusing, calculated from Equation 3,using Equation 6.

FIG. 3 is a drawing illustrating a plurality of apertures according toan embodiment of the present disclosure.

Referring to FIG. 3, at least one aperture 310 among a plurality ofapertures 310 and 320 according to an embodiment of the presentdisclosure may have a central location which is misaligned with acentral location of the other aperture 320 except for the at least oneaperture 310 among the plurality of apertures 310 and 320. Hereinafter,an embodiment is exemplified as the plurality of apertures 310 and 320are configured with the two apertures. However, embodiments are notlimited thereto. For example, the plurality of apertures 310 and 320 maybe configured with three apertures, four apertures, or the like. Adetailed description for this will be given with reference to FIG. 4.

Further, the plurality of apertures 310 and 320 may be formed on oneoptical system. Particularly, the at least one aperture 310 among theplurality of apertures 310 and 320 may be formed on the other aperture320.

In this case, the other aperture 320 may be formed by including a filterwhich detects a specific optical signal in the optical system in whichthe plurality of apertures 310 and 320 are formed, and the at least oneaperture 310 may be formed by etching a specified region on the otheraperture 320. For example, the other aperture 320 may be formed bycoating a filter which blocks an optical signal of greater than or equalto a wavelength of 650 nm on a rear surface of an optical system whichis formed of a glass plate and coating a filter which blocks an opticalsignal of greater than or equal to a wavelength of 810 nm on a frontsurface of the optical system which is formed of the glass plate.Further, the at least one aperture 310 may be formed by etching aspecific region on the other aperture 320 to have a central locationwhich is misaligned with a central location of the other aperture 320.

Thus, the at least one aperture 310 may introduce an optical signal of adifferent wavelength from a wavelength of an optical signal introducedby the other aperture 320. For example, the optical signal introducedthrough the other aperture 320 may be an optical signal of a visiblewavelength, and the optical signal introduced through the at least oneaperture 310 may be an optical signal of a visible wavelength and anoptical signal of an infrared wavelength.

In this case, each of the plurality of apertures may be formed invarious forms to have one of a circle, a triangle, a quadrangle, or apolygon.

Further, the optical system in which the plurality of apertures 310 and320 are formed may be adaptively located on an upper or lower portion ofa lens included in a camera system.

FIGS. 4A to 4C are drawings illustrating a plurality of aperturesaccording to another embodiment of the present disclosure.

Referring to FIGS. 4A to 4C, the plurality of apertures included in acamera system according to an embodiment of the present disclosure mayhave various number of apertures included in the plurality of aperturesor various forms of apertures included in the plurality of apertures.

For example, as shown in reference numeral 410, the plurality ofapertures included in the camera system may be configured with a firstaperture which introduces a red-green (RG) signal and a second aperturewhich introduces a blue (B) signal, the first aperture and the secondaperture being formed on one optical system. Further, as shown inreference numeral 420, the plurality of apertures may be configured witha first aperture which introduces an RGB signal and a second aperturewhich introduces an infrared (IR) signal.

The plurality of apertures are not restricted or limited to FIGS. 4A to4C and may be configured with various forms of various number ofapertures.

FIG. 5 is a flowchart illustrating an auto focusing method of a camerasystem according to an embodiment of the present disclosure.

Referring to FIG. 5, in operation 510, the camera system according to anembodiment of the present disclosure may generate a plurality of imagesby processing an optical signal introduced through each of a pluralityof apertures.

Herein, at least one aperture among the plurality of apertures may havea central location which is misaligned with a central location of theother aperture except for the at least one aperture among the pluralityof apertures. In this case, the at least one aperture may introduce anoptical signal of a different wavelength from a wavelength of an opticalsignal introduced by the other aperture.

Further, the plurality apertures may be formed on one optical system.Particularly, the at least one aperture among the plurality of aperturesmay be formed on the other aperture except for the at least one apertureamong the plurality of apertures.

For example, the other aperture may be formed by including a filterwhich detects a specific optical signal in the optical system where theplurality of apertures are formed, and the at least one aperture may beformed by etching a specific region on the other aperture.

Further, each of the plurality of apertures may be formed to have one ofa circle, a triangle, a quadrangle, or a polygon.

In operation 510, the camera system may process the plurality of imagesand may calculate parameters for auto focusing. For example, the camerasystem may calculate a disparity between central locations of theplurality of images, a distance between a central location of the atleast one aperture and a central location of the other aperture, adistance between the optical system in which the plurality of aperturesare formed and an image sensor, a subject distance which is focused onthe image sensor, a focal distance, and the like, as the parameters forthe auto focusing.

In operation 520, the camera system may determine a moving distance ofthe image sensor using the characteristics of the plurality of aperturesand the plurality of images for auto focusing.

In detail, the camera system may determine a moving distance of theimage sensor using the characteristics of the plurality of apertures,based on a difference between central locations of the plurality ofimages, a distance between a central location of the at least oneaperture and a central location of the other aperture, a distancebetween the optical system in which the plurality of apertures areformed and the image sensor, a subject distance which is focused on theimage sensor, and a focal distance, which are obtained from theplurality of images.

Further, although not illustrated in FIG. 5, the camera system mayverify the distance of the image sensor using the characteristics of theplurality of apertures and a difference between blur levels in theplurality of images.

Thus, the camera system may focus by moving the image sensor or maymoving a lens barrel in which the optical system where the plurality ofapertures are formed is fixed to relatively move the mage sensor, basedon the distance determined in operation 520.

FIG. 6 is a block diagram illustrating a camera system having an autofocusing function according to an embodiment of the present disclosure.

Referring to FIG. 6, the camera system according to an embodiment of thepresent disclosure may include a plurality of apertures 610, an imagesensor 620, and an auto focusing unit 630.

Herein, at least one aperture among the plurality of apertures 610 mayhave a central location which is misaligned with a central location ofthe other aperture except for the at least one aperture among theplurality of apertures 610. In this case, the at least one aperture mayintroduce an optical signal of a different wavelength from a wavelengthof an optical signal introduced by the other aperture.

Further, the plurality of apertures 610 may be formed on one opticalsystem. Particularly, the at least one aperture among the plurality ofapertures 610 may be formed on the other aperture except for the atleast one aperture among the plurality of apertures 610.

For example, the other aperture may be formed by including a filterwhich detects a specific optical signal in the optical system where theplurality of apertures 610 are formed, and the at least one aperture maybe formed by etching a specific region on the other aperture.

Further, each of the plurality of apertures 610 may be formed to haveone of a circle, a triangle, a quadrangle, or a polygon.

The image sensor 620 may generate a plurality of images by processing anoptical signal introduced through each of the plurality of apertures610.

In this case, the image sensor 620 may process the plurality of imagesand may calculate parameters for auto focusing. For example, the imagesensor 620 may calculate a disparity between central locations of theplurality of images, a distance between a central location of the atleast one aperture and a central location of the other aperture, adistance between the optical system in which the plurality of apertures610 are formed and the image sensor 620, a subject distance which isfocused on the image sensor 620, a focal distance, and the like, as theparameters for the auto focusing.

The auto focusing unit 630 may determine a moving distance of the imagesensor 620 using the characteristics of the plurality of apertures 610and the plurality of images for auto focusing.

In detail, the auto focusing unit 630 may determine a moving distance ofthe image sensor 620 using the characteristics of the plurality ofapertures 610, based on a difference between central locations of theplurality of images, a distance between the central location of the atleast one aperture and the central location of the other aperture, adistance between the optical system in which the plurality of apertures610 are formed and the image sensor 620, a subject distance which isfocused on the image sensor 620, and a focal distance, which areobtained from the plurality of images.

Further, although not illustrated in FIG. 6, the auto focusing unit 630may verify the moving distance of the image sensor 620 using thecharacteristics of the plurality of apertures 610 and a differencebetween blur levels in the plurality of images.

Thus, the auto focusing unit 630 may focus by moving the image sensor620 or may moving a lens barrel in which the optical system where theplurality of apertures 610 are formed is fixed to relatively move themage sensor 620, based on the determined distance.

FIG. 7 is a flowchart illustrating a depth estimation method of a camerasystem according to an embodiment of the present disclosure.

Referring to FIG. 7, in operation 710, the camera system according to anembodiment of the present disclosure may generate a plurality of imagesby processing an optical signal introduced through each of a pluralityof apertures.

Herein, at least one aperture among the plurality of apertures may havea central location which is misaligned with a central location of theother aperture except for the at least one aperture among the pluralityof apertures. In this case, the at least one aperture may introduce anoptical signal of a different wavelength from a wavelength of an opticalsignal introduced by the other aperture.

Further, the plurality apertures may be formed on one optical system.Particularly, the at least one aperture among the plurality of aperturesmay be formed on the other aperture except for the at least one apertureamong the plurality of apertures.

For example, the other aperture may be formed by including a filterwhich detects a specific optical signal in the optical system where theplurality of apertures are formed, and the at least one aperture may beformed by etching a specific region on the other aperture.

Further, each of the plurality of apertures may be formed to have one ofa circle, a triangle, a quadrangle, or a polygon.

In operation 710, the camera system may process the plurality of imagesand may calculate parameters for depth estimation. For example, thecamera system may calculate a disparity between central locations of theplurality of images, a distance between a central location of the atleast one aperture and a central location of the other aperture, asubject distance which is focused on an image sensor, a focal distance,and the like, as the parameters for the depth estimation.

In operation 720, the camera system may estimate a depth from a subjectto the optical system in which the plurality of apertures are formed,using the plurality of images.

In detail, the camera system may estimate the depth from the subject tothe optical system in which the plurality of apertures are formed, basedon a disparity between the central locations of the plurality of images,a distance between the central location of the at least one aperture andthe central location of the other aperture, a subject distance which isfocused on the image sensor, and a focal distance, which are obtainedfrom the plurality of images.

Further, although not illustrated in FIG. 7, the camera system mayverify the depth from the subject to the optical system in which theplurality of apertures are formed, using a disparity between blur levelsin the plurality of images.

FIG. 8 is a block diagram illustrating a camera system having a depthestimation function according to an embodiment of the presentdisclosure.

Referring to FIG. 8, the camera system according to an embodiment of thepresent disclosure may include a plurality of apertures 810, an imagesensor 820, and a depth estimation unit 830.

Herein, at least one aperture among the plurality of apertures 810 mayhave a central location which is misaligned with a central location ofthe other aperture except for the at least one aperture among theplurality of apertures 810. In this case, the at least one aperture mayintroduce an optical signal of a different wavelength from a wavelengthof an optical signal introduced by the other aperture.

Further, the plurality apertures 810 may be formed on one opticalsystem. Particularly, the at least one aperture among the plurality ofapertures 810 may be formed on the other aperture except for the atleast one aperture among the plurality of apertures 810.

For example, the other aperture may be formed by including a filterwhich detects a specific optical signal in the optical system where theplurality of apertures 810 are formed, and the at least one aperture maybe formed by etching a specific region on the other aperture.

Further, each of the plurality of apertures 810 may be formed to haveone of a circle, a triangle, a quadrangle, or a polygon.

The image sensor 820 may generate a plurality of images by processing anoptical signal introduced through each of the plurality of apertures810.

In this case, the image sensor 820 may process the plurality of imagesand may calculate parameters for depth estimation. For example, theimage sensor 820 may calculate a disparity between central locations ofthe plurality of images, a distance between a central location of the atleast one aperture and a central location of the other aperture, asubject distance which is focused on the image sensor 820, a focaldistance, and the like, as the parameters for the depth estimation.

The depth estimation unit 830 may estimate a depth from a subject to theoptical system in which the plurality of apertures are formed, using theplurality of images.

In detail, the depth estimation unit 830 may estimate the depth from thesubject to the optical system in which the plurality of apertures areformed, based on a disparity between the central locations of theplurality of images, a distance between the central location of the atleast one aperture and the central location of the other aperture, asubject distance which is focused on the image sensor 820, and a focaldistance, which are obtained from the plurality of images.

Further, although not illustrated in FIG. 8, the depth estimation unitmay verify the depth from the subject to the optical system in which theplurality of apertures 810 are formed, using a disparity between blurlevels in the plurality of images.

MODE FOR INVENTION

While a few exemplary embodiments have been shown and described withreference to the accompanying drawings, it will be apparent to thoseskilled in the art that various modifications and variations can be madefrom the foregoing descriptions. For example, adequate effects may beachieved even if the foregoing processes and methods are carried out indifferent order than described above, and/or the aforementionedelements, such as systems, structures, devices, or circuits, arecombined or coupled in different forms and modes than as described aboveor be substituted or switched with other components or equivalents.

Therefore, other implements, other embodiments, and equivalents toclaims are within the scope of the following claims.

The invention claimed is:
 1. A multi-aperture camera system having anauto focusing function, the system comprising: a plurality of apertures;an image sensor configured to generate a plurality of images byprocessing an optical signal introduced through each of the plurality ofapertures; and an auto focusing unit configured to determine a distancethe image sensor moves using the characteristics of the plurality ofapertures and the plurality of images for auto focusing, wherein atleast one aperture among the plurality of apertures has a centrallocation which is misaligned with a central location of the otheraperture, wherein the auto focusing unit determines a distance where theimage sensor moves relative to the plurality of apertures, based on adifference between central locations of the plurality of images, adistance between a central location of the at least one aperture and acentral location of the other aperture, a distance between an opticalsystem in which the plurality of apertures are formed and the imagesensor, a subject distance which is focused on the image sensor, and afocal distance.
 2. The system of claim 1, wherein the plurality ofapertures are formed on one optical system.
 3. The system of claim 1,wherein the at least one aperture is formed on the other aperture. 4.The system of claim 3, wherein the other aperture js formed by includinga filter which detects a specific optical signal in an optical system inwhich the plurality of apertures are formed.
 5. The system of claim 4,wherein the at least one aperture is formed by etching a specific regionon the other aperture.
 6. The system of claim 1, wherein the at leastone aperture and the other aperture are formed to have one of a circle,a triangle, a quadrangle, or a polygon.
 7. The system of claim 1,wherein the at least one aperture introduces an optical signal of adifferent wavelength from a wavelength of an optical signal introducedby the other aperture.
 8. The system of claim 1, wherein the autofocusing unit verifies the distance where the image sensor movesrelative to the plurality of apertures, using a difference between blurlevels in the plurality of images.